Identifying risk of death in children hospitalized with community-acquired pneumonia

Abstract Objective To externally validate a tool developed by the Pneumonia Research Partnership to Assess WHO Recommendations study group for identification of the risk of death in children hospitalized with community-acquired pneumonia, the PREPARE tool. Methods We did a secondary analysis of data collected during hospital-based surveillance of children with community-acquired pneumonia in northern India from January 2015 to February 2022. We included children aged 2–59 months with pulse oximetry assessment. We used multivariable backward stepwise logistic regression analysis to assess the strength of association of the PREPARE variables (except hypothermia) with pneumonia-related death. We estimated sensitivity, specificity, and positive and negative likelihood ratios of the PREPARE score at cut-off scores ≥ 3, ≥ 4 and ≥ 5. Findings Of 10 943 children screened, 6745 (61.6%) were included in our analysis, of whom 93 (1.4%) died. Age of < 1 year, female sex, weight-for-age < −3 standard deviations, respiratory rate of ≥ 20 breaths/min higher than the age-specific cut-off, and lethargy, convulsions, cyanosis and blood oxygen saturation < 90% were associated with death. In the validation, the PREPARE score had the highest sensitivity (79.6%) with concurrent highest specificity (72.5%) to identify hospitalized children at risk of death from community-acquired pneumonia at a cut-off score of ≥ 5. Area under curve was 0.82 (95% confidence interval: 0.77–0.86). Conclusion The PREPARE tool with pulse oximetry showed good discriminatory ability on external validation in northern India. The tool can be used to assess risk of death of hospitalized children aged 2–59 months with community-acquired pneumonia for early referral to higher-level facilities.


Introduction
Community-acquired pneumonia is the leading cause of death in children younger than 5 years, accounting for 14.2% (0.74/5.2 million) of deaths in this age group worldwide in 2019. 1,2 In 2018, 0.8 million children globally died from community-acquired pneumonia, of which 0.13 million were in India, that is about 14 deaths every hour. 3 The World Health Organization (WHO) has classified community-acquired pneumonia as: (i) pneumonia, where there is fast breathing with or without chest indrawing; and (ii) severe pneumonia, where one or more danger signs are present, such as inability to drink, persistent vomiting, convulsions, lethargy or unconsciousness, stridor in a calm child or severe malnutrition. 4 Most deaths related to community-acquired pneumonia occur in cases of severe pneumonia. 5 Therefore, patients with severe community-acquired pneumonia must be hospitalized for optimal care, ideally in a tertiary-care facility. Generally, families opt for a facility closest to their home for treatment of illness. 6 This choice increases the risk of hospital mortality. Therefore, in 2022, the Pneumonia Research Partnership to Assess WHO Recommendations (PREPARE) study group developed a tool to help identify the risk of hospital deaths in patients with severe community-acquired pneumonia (the PREPARE tool). 7 In developing this tool, data from 27 388 children aged 2-59 months from low-and middle-income countries were used. The tool has two scores, one with pulse oximetry and another without pulse oximetry. Both scores have good sensitivity and specificity to identify children at risk of death at specific cutoff points, and good discriminatory ability. 7 The tool uses data that are routinely collected in most hospitals. The PREPARE tool is the latest of several tools developed to identify patients with community-acquired pneumonia at risk of death. [8][9][10][11] However, before the widespread use of the PREPARE tool is promoted, external validation needs to be done. Therefore, the objective of our study was to externally validate this tool with pulse oximetry for identification of children aged 2-59 months with community-acquired pneumonia at risk of death.

Method Study design
We used data collected in a prospective, multisite ongoing hospital-based surveillance system on community-acquired pneumonia in children aged 2-59 months. The surveillance included four districts, namely Lucknow and Etawah in Uttar Pradesh, and Patna and Darbhanga in Bihar, India, and started on 1 January 2015. 12 The system included children hospitalized for community-acquired pneumonia in public and private hospitals. We applied the PREPARE tool retrospectively to these data to assess its ability to predict hospital mortality related to pneumonia. The surveillance system had collected all the data used by the tool except for two variables: lower chest indrawing, and body temperature measured by thermometer. Therefore, for our conservative external validation, we did not include these two variables in determining the PREPARE score. We used the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis checklist for the external validation of the PREPARE tool. 13

Sample size
For an expected sensitivity and specificity of 70% 7 of the PREPARE score on external validation, with 10% absolute Objective To externally validate a tool developed by the Pneumonia Research Partnership to Assess WHO Recommendations study group for identification of the risk of death in children hospitalized with community-acquired pneumonia, the PREPARE tool. Methods We did a secondary analysis of data collected during hospital-based surveillance of children with community-acquired pneumonia in northern India from January 2015 to February 2022. We included children aged 2-59 months with pulse oximetry assessment. We used multivariable backward stepwise logistic regression analysis to assess the strength of association of the PREPARE variables (except hypothermia) with pneumonia-related death. We estimated sensitivity, specificity, and positive and negative likelihood ratios of the PREPARE score at cut-off scores ≥ 3, ≥ 4 and ≥ 5. Findings Of 10 943 children screened, 6745 (61.6%) were included in our analysis, of whom 93 (1.4%) died. Age of < 1 year, female sex, weight-for-age < −3 standard deviations, respiratory rate of ≥ 20 breaths/min higher than the age-specific cut-off, and lethargy, convulsions, cyanosis and blood oxygen saturation < 90% were associated with death. In the validation, the PREPARE score had the highest sensitivity (79.6%) with concurrent highest specificity (72.5%) to identify hospitalized children at risk of death from community-acquired pneumonia at a cut-off score of ≥ 5. Area under curve was 0.82 (95% confidence interval: 0.77-0.86). Conclusion The PREPARE tool with pulse oximetry showed good discriminatory ability on external validation in northern India. The tool can be used to assess risk of death of hospitalized children aged 2-59 months with community-acquired pneumonia for early referral to higher-level facilities.

Research
Validation of pneumonia mortality risk tool Shally Awasthi et al.
precision and 95% confidence interval (CI) with a 1.4% prevalence of hospital mortality, the minimum required sample was 5763. 14

Data source
We collected data on children hospitalized with community-acquired pneumonia aged 2-59 months from January 2015 to February 2022 from the hospitals-based network in the selected districts. In the original surveillance, trained surveillance officers recruited children with community-acquired pneumonia from the network hospitals after parental consent. Inclusion criteria were: (i) 2-59 months of age; (ii) hospitalization with symptoms of community-acquired pneumonia as defined by WHO; (iii) resident of the project district; (iv) illness of < 14 days; (v) no previous hospitalization for community-acquired pneumonia nor recruitment in the surveillance; and (vi) parental consent to participate. We excluded children without pulse oximetry data from our analysis.

Variables
The sociodemographic and clinical variables included were age, sex, weight-forage, immunization status, respiratory rate, lethargy and/or unconsciousness, comorbidities, convulsions, cyanosis and blood oxygen saturation. Our outcome measure was hospital death from community-acquired pneumonia. Comorbidities included congenital heart disease and a history of fast breathing, and having a cough three times or more in 6 months as a surrogate marker of asthma. We defined tachypnoea as (greater than/equal to) ≥ 50 breaths/ min for children aged 2-11 months and (greater than/equal to) ≥ 40 breaths/ min for children aged 12-59 months. 4,15 We categorized nutrition status of the children as: severe malnutrition with weight-for-age z scores < −3 standard deviation (SD); moderate malnutrition with weight-for-age z scores −3 SD to −2 SD; and adequate nutrition as weight-for-age z scores > −2 SD. 16 We categorized blood oxygen saturation of < 90% as severe hypoxemia, 90-92% as mild hypoxemia and 93-100% as normal blood oxygen. All variables included in our analysis had been recorded at initial enrolment in the surveillance system. All deaths included in our analysis occurred during the hospital stay for this episode of community-acquired pneumonia.
We categorized children as fully immunized if they had received all vaccines as per the national immunization schedule of India in their first year of life. 17 Because pneumococcal conjugate vaccine was rolled out in a phased manner from 1 June 2017 onwards but staggered during the coronavirus disease 2019 (COVID-19) pandemic, we did not include this vaccine in the definition of fully vaccinated. 18,19 The PREPARE tool has one score that includes hypoxia, measured by pulse oximetry, which ranges from 0 to 17 and another score that does not include hypoxia, which ranges from 0 to 20. 7 The second score allows resourceconstrained areas with no access to pulse oximetry and skilled human resources to also use the PREPARE tool. The primary data from the surveillance system did not include the presence of lower chest indrawing as this symptom was not used in the revised WHO classification of community-acquired pneumonia. 5 Since lack of this information does not affect the coding in the PREPARE score with pulse oximetry, we only included Validation of pneumonia mortality risk tool Shally Awasthi et al. children whose pulse oximetry had been measured. As we did not have data on body temperature, our PREPARE risk assessment score ranged from 0 to 14 (three points for the presence of hypothermia were not included). We excluded children who had data missing on any of the variables we used in our scoring.

Data analysis
We compared sociodemographic and clinical variables of hospitalized children who died of community-acquired pneumonia and those who survived the episode. We used numbers and percentages for categorical variables. We converted continuous variables into categorical variables based on recommended thresholds before developing our model to facilitate external validation of the PREPARE tool.
We used multivariable backward stepwise logistic regression analysis to identify risk factors for hospital mortality for community-acquired pneumonia. We included variables with a P-value ≤ 0.1 in the univariable analysis in the regression analysis. We calculated adjusted odds ratios (aOR) and 95% CIs of the association of sociodemographic and clinical variables with hospital mortality. This model also assessed the comparability of our data with the PREPARE model with pulse oximetry. We considered a two-tailed P-value of < 0.05 as statistically significant.
We used cut-off points of ≥ 3, ≥ 4 and ≥ 5 in PREPARE scores in our sensitivity and specificity analysis of the PREPARE tool, as reported in the development and internal validation of the tool. 7 We calculated positive and negative likelihood ratios with 95% CI and the Youden index to assess the improvement in likelihood of correctly identifying hospital mortality. We constructed receiver operating characteristic curves for the PREPARE tool with pulse oximetry. We used area under the curve with 95% CI to assess the discriminatory ability of score. The scale used to qualify the discriminatory ability of score was area under the curve ≥ 0.90 for excellent discrimination, area under the curve 0.80 to 0.89 for good discrimination, area under the curve 0.70 to 0.79 for fair discrimination and area under the curve < 0.70 for poor dis-crimination. 20,21 We used SPSS version 24 (SPSS Inc., Chicago, United States of America) for all statistical analyses.

Ethical considerations
The primary surveillance study was approved by the Ethics Review Committee of each participating site: (

Results
From January 2015 to February 2022, 10 943 children were screened in the four districts. We excluded 131 children with no parental consent and 4067 children  Comparison of sociodemographic and clinical variables of hospitalized children who survived and who died is given in Table 1. Significant differences in mortality were found with all variables (all P < 0.001), except age group.
Multivariable backward stepwise logistic regression analysis of sociodemographic and clinical variables associated with hospital mortality is shown in Table 2. Female sex, weight-for-age < −3 SD, respiratory rate ≥ 20 breaths more than the age-specific cut-off, lethargy and/or unconsciousness, convulsions, cyanosis and blood oxygen saturation < 90% were all significantly associated with death.
The PREPARE tool had the highest concurrent sensitivity and specificity to identify hospital mortality at a cut-off score of ≥ 5: 79.6% sensitivity and 72.5% specificity with a positive likelihood ratio of 2.90 (95% CI: 2.59-3.23) and negative likelihood ratio of 0.28 (95% CI: 0.19-0.42; Table 3). The Youden index showed moderate discriminatory power (0.52) at a score of ≥ 5. The overall area under the curve of the PREPARE risk score was 0.82 (95% CI: 0.77-0.86; Fig. 1). The area under the curve of the receiver operating characteristic curve using mortality and PREPARE score at a cut-off of ≥ 5 was 0.71 (95% CI: 0.65-0.77) indicating fair discriminatory power.
Comparison of the PREPARE tool to other risk assessment tools -such as the Respiratory Index of Severity in Children (RISC) from South Africa, 8 the Modified Respiratory Index of Severity in Children (mRISC) from Kenya, 9 RISC-Malawi from Malawi, 10 and Pneumonia Etiology Research for Child Health (PERCH) from Bangladesh, Gambia, Kenya, Mali, South Africa, Thailand and Zambia 11 -that have been developed to identify patients with pneumonia at risk of death during hospital stay is shown in Table 4. Even though the tools have different predic-tor variables, weight-for-age z-score is used in all of them. Oxygen saturation is included in all except mRISC, unconsciousness is included in all except the PERCH tool, and sex is not included in the RISC and mRISC tools. All of these variables are included in the PREPARE tool. The RISC score had the highest area under the curve. However, this score is based on children aged 0-24 months whereas community-acquired pneumonia affects children up to 5 years. 1 As such, the RISC tool has limited use. The PREPARE tool was developed for children aged 2-59 months using data from 20 countries which makes it more generalizable.

Discussion
Our external validation showed that the PREPARE tool with pulse oximetry had good discriminatory ability and the sensitivity and specificity of the tool was highest at a cut-off value of ≥ 5.
Internal validation of the PREPARE score reported an area under the curve of 0.83 (95% CI: 0.81-0.84) for hospi-  In our external validation, the odds of death for all included variables except age were similar to the reported internal validation of the PREPARE tool. 7 Children aged 2-5 months had statistically higher odds of death on internal validation of the PREPARE tool. 7 We did not have adequate sample sizes to analyse the mortality differences between the age groups.
Immunization is an important intervention for the prevention of community-acquired pneumonia in children. 22 The most common etiological agents are Haemophilus influenzae and Streptococcus pneumoniae. 23 The H. influenzae vaccine was introduced as a part of pentavalent vaccine in India's national immunization schedule in 2011 in phases. The phased roll-out of the pneumococcal conjugate vaccine started in 2017, so some of the data used in our study were collected before the introduction of this vaccine. The data used in the development of the PREPARE tool were collected before the introduction of the H. influenzae and pneumococcal conjugate vaccines in some of the 20 countries from which data were obtained. Therefore, immunization status is not included in the PREPARE tool, although we found immunization to be associated with hospital mortality of community-acquired pneumonia. However, often parents do not have immunization records with them when their child is admitted to hospital, and hence exclusion of immunization seems justified.
Body temperature is a risk factor included in the PREPARE tool as a strong association between hospital mortality and hypothermia has been shown. 7 Hypothermia has been reported to be associated with sepsis and excess mortality. 24 Body temperature was not recorded in our data set and hence we could not include it in our external validation of the PREPARE tool. However, even without the inclusion of body temperature, the PREPARE tool showed good test characteristics in our validation, similar to the internal validation. 7 Therefore, the PREPARE tool with pulse oximetry appears to be an effective tool without data on body temperature.
Chest indrawing was found to have no association with mortality in the PREPARE tool with pulse oximetry and was not assessed in our study. This symptom is included in the RISC and mRISC tools but not in the PERCH and RISC Malawi tools for pneumonia-related mortality in hospitalized patients. The classification and management of community-acquired pneumonia was revised in the 2014 WHO guidelines and chest indrawing was not included. In younger children chest indrawing is a less specific finding because they have more compliant chest walls. However, chest indrawing along with signs of severe respiratory distress calls for attention and intervention. 25 Therefore, in the PREPARE tool without pulse oximetry, which we did not externally validate, the presence of lower chest indrawing is given a score of +1.
We applied the PREPARE tool using data with pulse oximetry. Hypoxia alone has been reported as a single risk factor for death from pneumonia in children. 26 We also found that the aOR of mortality in children with hypoxia (blood oxygen saturation < 90%) was 4.02 (95% CI: 2.43-6.65). A systematic review and meta-analysis supported routine evaluation of blood oxygen saturation to identify children with acute lower respiratory infection at increased risk of death. 27 In line with this recommendation, hypoxemia is incorporated in tools such as RISC, RISC-Malawi and PERCH. Since the COVID-19 pandemic, many health facilities, including peripheral health centres, are using pulse oximeters. We recommend the use of pulse oximetry to document hypoxemia in patients with community-acquired pneumonia. 28 Our study has some limitations. The data used in our study started to be collected in 2015, before the PRE-PARE tool was developed. Hence, not all variables included in the original PREPARE tool were available in our data set, specifically body temperature and chest indrawing. Hypothermia has been strongly associated with mortality in the PREPARE internal validation study and hence needs further external validation. As mentioned earlier, chest indrawing lacked association with mortality in the internal validation of the PREPARE tool with pulse oximetry. Some of the data in our study were collected before the introduction of pneumococcal conjugate vaccine and hence mortality may differ in populations with different vaccination coverage. Since our data were collected for another study and several sites were involved, inaccuracies in data collection could have led to misclassification bias of the variables used in the PREPARE tool. We did not validate the PREPARE tool without pulse oximetry as we did not have the necessary sample size for (. . .continued) Validation of pneumonia mortality risk tool Shally Awasthi et al.
70% sensitivity and specificity as found in the internal validation of the PRE-PARE tool. 7 We used data from northern India only for our validation. Validation of the PREPARE tool in different parts of India and other regions is needed before incorporating it in standard of care guidelines. Similarly, external validation of the PREPARE tool without pulse oximetry should be done.